Time‐varying measures of cerebral network centrality correlate with visual saliency during movie watching

Abstract The extensive development of graph‐theoretic analysis for functional connectivity has revealed the multifaceted characteristics of brain networks. Network centralities identify the principal functional regions, individual differences, and hub structure in brain networks. Neuroimaging studies using movie‐watching have investigated brain function under naturalistic stimuli. Visual saliency is one of the promising measures for revealing cognition and emotions driven by naturalistic stimuli. This study investigated whether the visual saliency in movies was associated with network centrality. The study examined eigenvector centrality (EC), which is a measure of a region's influence in the brain network, and the participation coefficient (PC), which reflects the hub structure in the brain, was used for comparison. Static and time‐varying EC and PC were analyzed by a parcel‐based technique. While EC was correlated with brain activity in parcels in the visual and auditory areas during movie‐watching, it was only correlated with parcels in the visual areas in the retinotopy task. In addition, high PC was consistently observed in parcels in the putative hub both during the tasks and the resting‐state condition. Time‐varying EC in the parietal parcels and time‐varying PC in the primary sensory parcels significantly correlated with visual saliency in the movies. These results suggest that time‐varying centralities in brain networks are distinctively associated with perceptual processing and subsequent higher processing of visual saliency.


OGAWA
Analyzing the brain activity associated with visual saliency is one of the potential measures for revealing cognitions and emotions driven by naturalistic stimuli Nguyen et al., 2017;for review, Sonkusare et al., 2019;Vanderwal et al., 2019).
The visual saliency of a scene characterizes spatially the strength of bottom-up features. A visual saliency map computed with a biologically plausible model can encode the conspicuity of the visual scene and provide a prediction of the attention deployment (Itti et al., 1998;Itti & Koch, 2001;Harel et al., 2007). Visual saliency is known to modulate activation in higher-order visual areas, and posterior parietal areas involved in visual attention (Gottlieb et al., 1998;Nardo et al., 2011Nardo et al., , 2014Capotosto et al., 2013;Santangelo & Macaluso, 2013;Santangelo et al., 2015). In addition to static scenes, visual saliency can serve to investigate the neural correlates of cognitive dynamics during movie-watching .
The characteristics of large-scale functional networks in the brain have been thoroughly investigated using graph-theoretic analysis (Bullmore & Sporns, 2009;Rubinov & Sporns, 2010;Fornito et al., 2013Fornito et al., , 2016Bassett & Sporns, 2017). During movie-watching, global and nodal graph indices characterize large-scale functional networks (Kim et al., 2017). Among the several graph-theoretic indices, network centrality identifies regions important for information processing in the brain network. While the eigenvector centrality (EC) indicates influential regions in the brain network both during tasks and in the resting state (Joyce et al., 2010;Lohmann et al., 2010;Zuo et al., 2012), the participation coefficient (PC) identifies the hub structure in the brain network Power et al., 2013;van den Heuvel & Sporns, 2013;Sporns, 2014). Therefore, high PC brain regions work as hubs for information processing, whereas high EC brain regions have a functional relevance for the brain network.
The large-scale neural activity patterns underlying cognitive processes and behaviors are associated with time-varying functional connectivity (Sakoglu et al., 2010;Hutchison et al., 2013;Leonardi et al., 2014;Leonardi & Van De Ville, 2015;Lurie et al., 2020). Centrality changes along with temporary changes in functional connectivity depending on task demands. Recent investigations in time-varying network centralities have revealed several brain network characteristics. Essentially, while the time-varying EC of the resting-state appears to reflect individual differences (Wink, 2019), time-varying PC demonstrates the changes in the network structure with increasing integration of various brain regions .
These centralities represent the diverse characteristics of the brain network.
This study hypothesized that time-varying centralities reveal brain network characteristics during movie-watching and examined whether time-varying EC and PC in the visual areas and posterior parietal regions correlated with visual saliency during movie-watching.
This study used fMRI data available from the public Human Connectome Project database (HCP, Ugurbil et al., 2013;Van Essen et al., 2013). Data pertaining to retinotopy task performance and the resting state were also analyzed for comparison. While the retinotopy task was expected to only evoke the activation of visual areas, no activation related to movie-watching was expected during the resting-state. HCP Parcellation (Glasser et al., 2016) was employed for centrality analysis.
A parcel corresponded to a node in the brain network. The static (time-averaged) and time-varying EC and PC during movie-watching, retinotopy task, and resting state were calculated parcel-wise. Parcelbased brain networks are preferred to voxel-wise centrality analysis for analyzing time-varying centrality since the latter involves a considerable amount of calculation load. Herein, the spatial map of PC was analyzed for consistency between movie-watching, experimental task, and resting state. Time-varying EC was used to evaluate the relationship between visual saliency and functional network profile during movie-watching.

Image processing
The image preprocessing details are described elsewhere Smith et al., 2013). Briefly, the following steps were performed: First, image susceptibility induced distortions were corrected.
The images were spatially normalized into the standard space of Montreal Neurology Institute coordinates. Next, the data were normal-

Network analyses
A proportional 7.5% threshold was applied to the correlation matrix to generate an adjacency matrix, which represented a binary undirected network (van den Heuvel et al., 2017; Figure 1). The adjacency matrix was used for calculating EC and PC. The threshold was determined based on the flow coefficient . As opposed to EC and PC, the flow coefficient is a local centrality metric. With respect to the network construction, the network nodes are expected to influence or communicate with each other more intensively. The mean flow coefficients across participants were calculated at 2.5%, 5.0%, 7.5%, 10.0%, 12.5%, and 15% proportions. The peak was found at 7.5%.
EC evaluates the importance of each network node-parcel- (Lohmann et al., 2010;Zuo et al., 2012). All connections in the adjacency matrix were used to calculate EC. Therefore, EC integrates the connectivity information about all functionally connected parcels. EC F I G U R E 1 Procedure for calculating centrality map and visual saliency in movies. (A) Procedure for calculating centrality maps. The denoised functional magnetic resonance imaging (fMRI) time-series are averaged in each parcel. Temporal correlation coefficients of the averaged time-series are calculated between parcels. A threshold is applied to the correlation matrix to generate the adjacency matrix where each element indicates the presence or absence of connection between parcels. Then, network centrality in each parcel is calculated and projected onto the standard brain surface to generate a centrality map. (B) Calculation of visual saliency in movies. The visual saliency map of each movie frame is calculated. The maps within each scan are spatially averaged to generate the mean visual saliency signal. A low-pass filter is applied to the mean visual saliency signal can be used to capture brain-wide large-scale characteristics. Thus, a high EC brain region is functionally crucial in the brain network (e.g. network resilience). The EC of parcel i was calculated as below: A is the adjacency matrix, λ 1 is the first eigenvalue, µ 1 is the first eigenvector, and N is the number of parcels. PC of parcel i was also calculated for the brain network (Guimerà & Nunes Amaral, 2005;Power et al., 2013): M is the total set of network communities estimated using Louvain community detection (Rubinov & Sporns, 2010), K i is the number of connections associated with parcel i, and K i (m) is the number of connections between parcel i and all parcels in community m. PC can be used to define a parcel acting as a connector hub between modules (i.e. local sub-networks) (Guimerà & Nunes Amaral, 2005;Rubinov & Sporns, 2010;Bertolero et al., 2015;Cohen & D'Esposito, 2016). While parcels with higher PC have connections across modules, parcels with lower PC tend to have connections within a module. In contrast to that of EC, the spatial profile of PC, that reflects the hub structure in the brain, was consistent between the tasks and resting-state condition.
Both time-varying eigenvector centrality (tEC) and participation coefficient (tPC) were calculated using the sliding window technique (Shakil et al., 2016). The window was set to 50 s (from before 25 s to after 25 s). The tEC and the tPC were calculated every second in time with the scan TR in MOVIE, except the first and the last 25 s. At time t, the adjacency matrix A(t) was calculated. Then, the tEC of parcel i at time t was calculated as below: The first eigenvalue λ 1 (t) and first eigenvector µ 1 (t) were calculated from the adjacency matrix A(t). The tPC of parcel i was also calculated as below: )2 The total set of network communities of time t, M(t), was estimated from the adjacency matrix A(t).

Visual saliency
The saliency toolbox (version 2.3, saliencytoolbox.net; Walther & Koch, 2006) was used to calculate the visual saliency map (Itti et al., 1998) of every movie frame ( Figure 1B). Visual saliency was calculated as follows. Each movie frame was decomposed into image features: intensity, color, and orientation. Feature maps were computed from the Gaussian pyramid procedure using the center-surround mechanism. Then, each feature map was normalized. Finally, the visual saliency map was obtained by combining the feature maps. The visual saliency map of each movie frame was 64 pixels in width and 45 pixels in height.
The visual saliency maps within each fMRI scan were spatially averaged across pixels and temporally averaged across frames to gen-erate the mean visual saliency signal over time for each scan, as follows:s F indicates the number of frames in time t (i.e. 24 frames), X indicates the number of pixels in the x-axis in the visual saliency map, and Y indicates the number of pixels in the y-axis in the visual saliency map. A low-pass filter (0.1 Hz) was applied to the mean visual saliency signal.
FMRI signals in visual areas were expected to reflect visual saliency.
The correlation between the fMRI signal in each parcel and the mean visual saliency signal was examined.

Parcel-wise relations between visual saliency and time-varying centralities
The temporal correlation between the mean visual saliency signal and tEC was calculated for each parcel in each participant. The temporal correlation between the mean visual saliency signal and tPC was also calculated. Subsequently, the correlation coefficients were transformed to Fisher-z value and statistically compared for each parcel.
Bonferroni correction was applied for the number of parcels, although the Bonferroni correction might be conservative and cause false negatives. Statistical threshold was set to Z > 4.3 (P-FWE < 0.01).

Centralities
EC and PC were calculated for each parcel, and for MOVIE, RET, and REST. Figure 3A shows the EC maps for MOVIE and RET. The  ECs in MOVIE were compared with those in RET and REST for each parcel. The parcels in visual areas showed higher EC for MOVIE compared with RET and REST ( Figure 4A,B, Table S2; paired t tests with Bonferroni correction for the number of parcels). The parcels in the inferior parietal lobule, lateral prefrontal cortex, and medial prefrontal cortex had higher EC for RET compared with MOVIE and REST (Figure S3A,B, Table S6). The parcels in the somatomotor network showed higher EC for REST compared with MOVIE and RET ( Figure S3C,D).
Likewise, PCs in MOVIE were compared with those in RET and REST for each parcel. Although the PC maps were similar among MOVIE, RET, and REST, the parcels in the default-mode network showed significantly higher PC for MOVIE compared with RET and REST (Figure 4C,D, Table S3; paired t tests with Bonferroni correction for the number of parcels). The other comparisons are shown in Figure S4 (see also Table S7).
We also examined whether the head motion influenced network centrality. The average head motion in each fMRI run was available from the HCP database (file: Movement_RelativeRMS_mean.txt). The mean head motion was calculated across fMRI runs for each participant in each condition. The correlation between mean head motion and network centralities across participants was calculated in each parcel.
The results showed low correlations between head motion and network centralities ( Figure S5).

Visual saliency
The correlation coefficients between the fMRI signal in each parcel and the mean visual saliency signal were Fisher-z transformed and statistically tested (one sample t test). As expected, the fMRI signals in the parcels in visual areas were significantly correlated with the mean visual saliency signal ( Figure 5A, Table S4; paired t-test with Bonferroni correction for the number of parcels). Significant correlations were also observed in parcels in the auditory areas. The visual saliency could emphasize the auditory signals. Figure 5B shows the group-averaged rmap of the same correlation. The averaged correlation coefficient was higher in parcels in the visual areas.

Relationship between visual saliency and time-varying eigenvector centrality and participation coefficient
The mean saliency signal was calculated for each TR scan and temporally filtered (Low-pass 0.1 Hz). Correlations between mean visual saliency signal, and tEC and tPC, were calculated for each parcel.
Fisher-z-transformed correlations were compared in each parcel (paired t test with Bonferroni correction for the number of parcels). The posterior parietal parcels showed a significant correlation between saliency and tEC ( Figure 6A, Table S5), while the primary sensory area parcels showed a significant correlation between saliency and tPC ( Figure 6B, Table S5).

DISCUSSION
This study investigated the association between time-varying network centrality in the brain and visual saliency during movie-watching. A parcel-based technique was used to construct a brain network from the functional connectivity between parcels. When comparing MOVIE and RET, the PC maps were similar, while the spatial distributions of EC were less similar. The correlation analysis between visual saliency and time-varying network centralities showed that tEC in the posterior parietal parcels, and tPC in the primary sensory areas, tracked visual saliency. These results suggest that tEC and tPC may be associated with different perceptual features of visual saliency.
The parcels in the posterior parietal cortex showed a significant correlation between tEC and visual saliency. Previous studies have demonstrated the role of the posterior parietal cortex in higher processing for visual saliency (Gottlieb et al., 1998;Arcizet et al., 2011;Santangelo et al., 2015;Chen et al., 2020). The results of this study suggest that large-scale brain networks are associated with visual saliency processing: The ventral attention network for stimulus-driven attentional capture (Yantis & Egeth, 1999) et al., 2013). Complementary to PC, which can detect a parcel acting as a connector hub between modules, within-module degree z-score can be used to detect a parcel with connections within a module (i.e. provincial hub) (Guimerà & Nunes Amaral, 2005;Rubinov & Sporns, 2010;Baum et al., 2017). To reveal the functional integration and segregation of brain networks, the combination of complemental network indices can be considered.
This study adopted a parcel-based technique to analyze the network centrality during movie-watching. Parcel-based analysis is becoming popular, not only for the resting state but for task-induced activation (Allan et al., 2019;Osada et al., 2019;Fujimoto et al., 2020;Suda et al., 2020). The main benefit of the parcel-based analysis, over voxel-wise or vertex-wise analysis of the whole brain, is lower computational load in both time and memory. However, one important limitation of the parcel-based analysis is its lower spatial resolution. Nevertheless, as a tool for analysis of the spatial distribution of network centrality in the whole brain, as in this case, the parcel-based analysis is more suitable than the vertex-wise analysis.
Time-varying network centrality may be beneficial for characterizing brain activity (Calhoun et al., 2014). The dynamics of network centricity calculated from functional connectivity are greatly affected by the nature of time-varying functional connectivity. In this study, the correlation between tEC and visual saliency was significant mainly in the posterior parietal cortex. In contrast, a higher EC was observed in the early visual cortex. The correlation between tPC and visual saliency was significant in primary sensory areas, whereas a higher PC was observed mainly in the prefrontal and posterior parietal cortices. Figure 6 shows significant parcels where centrality changes as visual saliency changes. tPC being higher in the visual cortex when visual saliency is also high likely reflects the fact that information is sent from the visual cortex to various areas when visual saliency is increased.
Thus, time-varying network centrality may not be related to the nature of static (or time-averaged) network centrality. Caution should be exercised in the interpretation of time-varying centrality, as its polarities are temporally not always equal to those of static network centrality .
Many resting state functional connectivity studies of brain networks use the Pearson's product-moment correlation coefficient and Fisher-z-transformation. When time series of the fMRI signals exhibit high temporal autocorrelation, the effective degrees of freedom are reduced and the standard error may be biased. Therefore, care should be taken when assessing the effective degrees of freedom influencing the network formation. The proportional threshold would be less sensitive to the effective degrees of freedom than the absolute threshold.
A solution would thus be to use the effective degrees of freedom for the Pearson's correlation that considers the temporal autocorrelation of the time series (Afyouni et al., 2019).
This study investigated the relationship between time-varying network centrality and visual saliency during movie-watching. The results showed a significant correlation between visual saliency and tEC in posterior parietal parcels and between visual saliency and tPC in primary sensory area parcels. The results also suggest that tEC is associated with spatial attention control in the parietal cortex evoked by visual saliency. These findings suggest that network centralities can be used for investigating brain activity characteristics in response to time-varying stimulus features, albeit with some caution in the interpretation.

ACKNOWLEDGMENTS
The authors like to thank Hironori Nakatani for his helpful comments on an earlier version of this manuscript. This study was supported by the Japan Society for the Promotion of Science (JSPS) KAKENHI (JP19K07807 and JP16H06324). Data were provided in part by the

COMPETING INTERESTS
The author declares no competing interests.

DATA AVAILABILITY STATEMENT
The centrality maps and the statistical results on the standard surface will be made available on reasonable request.